Letter frequency is the number of times

letters Letter, letters, or literature may refer to: Characters typeface * Letter (alphabet), a character representing one or more of the sounds used in speech; any of the symbols of an alphabet. * Letterform, the graphic form of a letter of the alphabe ...

of the

alphabet An alphabet is a standardized set of basic written graphemes (called letters) that represent the phonemes of certain spoken languages. Not all writing systems represent language in this way; in a syllabary, each character represents a syllab ...

appear on average in

written language A written language is the representation of a spoken or gestural language by means of a writing system. Written language is an invention in that it must be taught to children, who will pick up spoken language or sign language by exposure eve ...

. Letter frequency analysis dates back to the

Arab The Arabs (singular: Arab; singular ar, عَرَبِيٌّ, DIN 31635: , , plural ar, عَرَب, DIN 31635: , Arabic pronunciation: ), also known as the Arab people, are an ethnic group mainly inhabiting the Arab world in Western Asia, ...

mathematician

Al-Kindi Abū Yūsuf Yaʻqūb ibn ʼIsḥāq aṣ-Ṣabbāḥ al-Kindī (; ar, أبو يوسف يعقوب بن إسحاق الصبّاح الكندي; la, Alkindus; c. 801–873 AD) was an Arab Muslim philosopher, polymath, mathematician, physician ...

(c. 801–873 AD), who formally developed the method to break

ciphers In cryptography, a cipher (or cypher) is an algorithm for performing encryption or decryption—a series of well-defined steps that can be followed as a procedure. An alternative, less common term is ''encipherment''. To encipher or encode i ...

. Letter frequency analysis gained importance in

Europe Europe is a large peninsula conventionally considered a continent in its own right because of its great physical size and the weight of its history and traditions. Europe is also considered a subcontinent of Eurasia and it is located entirel ...

with the development of

movable type Movable type (US English; moveable type in British English) is the system and technology of printing and typography that uses movable components to reproduce the elements of a document (usually individual alphanumeric characters or punctuation m ...

in 1450 AD, where one must estimate the amount of type required for each

letterform A letterform, letter-form or letter form, is a term used especially in typography, palaeography, calligraphy and epigraphy to mean a letter (alphabet), letter's shape. A letterform is a type of glyph, which is a specific, concrete way of writing ...

. Linguists use letter frequency analysis as a rudimentary technique for

language identification In natural language processing, language identification or language guessing is the problem of determining which natural language given content is in. Computational approaches to this problem view it as a special case of text categorization, solv ...

, where it is particularly effective as an indication of whether an unknown

writing system A writing system is a method of visually representing verbal communication, based on a script and a set of rules regulating its use. While both writing and speech are useful in conveying messages, writing differs in also being a reliable fo ...

is alphabetic,

syllabic Syllabic may refer to: *Syllable, a unit of speech sound, considered the building block of words **Syllabic consonant, a consonant that forms the nucleus of a syllable *Syllabary, writing system using symbols for syllables *Abugida, writing system ...

, or

ideographic An ideogram or ideograph (from Greek "idea" and "to write") is a graphic symbol that represents an idea or concept, independent of any particular language, and specific words or phrases. Some ideograms are comprehensible only by familiari ...

. The use of letter frequencies and frequency analysis plays a fundamental role in

cryptograms A cryptogram is a type of puzzle that consists of a short piece of encrypted text. Generally the cipher used to encrypt the text is simple enough that the cryptogram can be solved by hand. Substitution ciphers where each letter is replaced by ...

and several word puzzle games, including Hangman, ''

Scrabble ''Scrabble'' is a word game in which two to four players score points by placing tiles, each bearing a single letter, onto a game board divided into a 15×15 grid of squares. The tiles must form words that, in crossword fashion, read left t ...

'', ''

Wordle ''Wordle'' is a web-based word game created and developed by Welsh software engineer Josh Wardle, and owned and published by The New York Times Company since 2022. Players have six attempts to guess a five-letter word, with feedback given for ...

'' and the television game show ''

Wheel of Fortune The Wheel of Fortune or ''Rota Fortunae'' has been a concept and metaphor since ancient times referring to the capricious nature of Fate. Wheel of Fortune may also refer to: Arts, entertainment, and media Art * ''The Wheel of Fortune'' (Burne-Jo ...

''. One of the earliest descriptions in classical literature of applying the knowledge of English letter frequency to solving a cryptogram is found in

Edgar Allan Poe Edgar Allan Poe (; Edgar Poe; January 19, 1809 – October 7, 1849) was an American writer, poet, editor, and literary critic. Poe is best known for his poetry and short stories, particularly his tales of mystery and the macabre. He is wid ...

's famous story ''

The Gold-Bug "The Gold-Bug" is a short story by American writer Edgar Allan Poe published in 1843. The plot follows William Legrand, who was bitten by a gold-colored bug. His servant Jupiter fears that Legrand is going insane and goes to Legrand's friend, an ...

'', where the method is successfully applied to decipher a message giving the location of a treasure hidden by

Captain Kidd William Kidd, also known as Captain William Kidd or simply Captain Kidd ( – 23 May 1701), was a Scottish sea captain who was commissioned as a privateer and had experience as a pirate. He was tried and executed in London in 1701 for murder a ...

Herbert S. Zim Herbert Spencer Zim (July 12, 1909 – December 5, 1994) was a naturalist, author, editor and educator best known as the founder (1945) and editor-in-chief of the Golden Guides series of nature books. Biography Zim was born 1909 in New York ...

, in his classic introductory cryptography text "Codes and Secret Writing", gives the English letter frequency sequence as "ETAON RISHD LFCMU GYPWB VKJXZQ", the most common letter pairs as "TH HE AN RE ER IN ON AT ND ST ES EN OF TE ED OR TI HI AS TO", and the most common doubled letters as "LL EE SS OO TT FF RR NN PP CC". Different ways of counting can produce somewhat different orders. Letter frequencies also have a strong effect on the design of some

keyboard layout A keyboard layout is any specific physical, visual or functional arrangement of the keys, legends, or key-meaning associations (respectively) of a computer keyboard, mobile phone, or other computer-controlled typographic keyboard. is the actua ...

s. The most frequent letters are on the bottom row of the

Blickensderfer typewriter The Blickensderfer Typewriter was invented by George Canfield Blickensderfer (1850–1917) and patented on August 4, 1891. Blickensderfer was the nephew of John Celivergos Zachos the inventor of the stenotype. Two models were initially un ...

, and the

home row Touch typing (also called blind typing, or touch keyboarding) is a style of typing. Although the phrase refers to typing without using the sense of sight to find the keys—specifically, a touch typist will know their location on the keyboard thr ...

of the

Dvorak keyboard layout Dvorak is a keyboard layout for English patented in 1936 by August Dvorak and his brother-in-law, William Dealey, as a faster and more ergonomic alternative to the QWERTY layout (the ''de facto'' standard keyboard layout). Dvorak proponent ...

Background

The frequency of letters in text has been studied for use in cryptanalysis, and frequency analysis in particular, dating back to the Arab mathematician Al-Kindi (c. 801–873 AD), who formally developed the method (the ciphers breakable by this technique go back at least to the Caesar cipher invented by

Julius Caesar Gaius Julius Caesar (; ; 12 July 100 BC – 15 March 44 BC), was a Roman general and statesman. A member of the First Triumvirate, Caesar led the Roman armies in the Gallic Wars before defeating his political rival Pompey in a civil war, ...

, so this method could have been explored in classical times). Letter frequency analysis gained additional importance in Europe with the development of movable type in 1450 AD, where one must estimate the amount of type required for each letterform, as evidenced by the variations in letter compartment size in typographer's type cases. No exact letter frequency distribution underlies a given language, since all writers write slightly differently. However, most languages have a characteristic distribution which is strongly apparent in longer texts. Even language changes as extreme as from

Old English Old English (, ), or Anglo-Saxon, is the earliest recorded form of the English language, spoken in England and southern and eastern Scotland in the early Middle Ages. It was brought to Great Britain by Anglo-Saxon settlers in the mid-5th c ...

to modern English (regarded as mutually unintelligible) show strong trends in related letter frequencies: over a small sample of Biblical passages, from most frequent to least frequent, of Old English compares to of modern English, with the most extreme differences concerning letterforms not shared.

Linotype machine The Linotype machine ( ) is a "line casting" machine used in printing; manufactured and sold by the former Mergenthaler Linotype Company and related It was a hot metal typesetting system that cast lines of metal type for individual uses. Lin ...

s for the English language assumed the letter order, from most to least common, to be based on the experience and custom of manual compositors. The equivalent for the French language was . Arranging the alphabet in Morse into groups of letters that require equal amounts of time to transmit, and then sorting these groups in increasing order, yields .

American Morse code American Morse Code — also known as Railroad Morse—is the latter-day name for the original version of the Morse Code developed in the mid-1840s, by Samuel Morse and Alfred Vail for their electric telegraph. The "American" qualifier was added ...

was developed in the 1830s by

Alfred Vail Alfred Lewis Vail (September 25, 1807 – January 18, 1859) was an American machinist and inventor. Along with Samuel Morse, Vail was central in developing and commercializing American telegraphy between 1837 and 1844. Vail and Morse were the f ...

, based on English-language letter frequencies, to encode the most frequent letters with the shortest symbols. Some efficiency was lost in the reformed version now used: the International Morse Code. Letter frequency was used by other telegraph systems, such as the

Murray Code The Baudot code is an early character encoding for telegraphy invented by Émile Baudot in the 1870s. It was the predecessor to the International Telegraph Alphabet No. 2 (ITA2), the most common teleprinter code in use until the advent of ASCII. ...

. Similar ideas are used in modern data-compression techniques such as

Huffman coding In computer science and information theory, a Huffman code is a particular type of optimal prefix code that is commonly used for lossless data compression. The process of finding or using such a code proceeds by means of Huffman coding, an algo ...

. Letter frequencies, like

word frequencies A word list (or ''lexicon'') is a list of a language's lexicon (generally sorted by frequency of occurrence either by levels or as a ranked list) within some given text corpus, serving the purpose of vocabulary acquisition. A lexicon sorted by ...

, tend to vary, both by writer and by subject. One cannot write an essay about x-rays without using frequently, and the essay will have an idiosyncratic letter frequency if the essay is about the use of x-rays to treat zebras in

Qatar Qatar (, ; ar, قطر, Qaṭar ; local vernacular pronunciation: ), officially the State of Qatar,) is a country in Western Asia. It occupies the Qatar Peninsula on the northeastern coast of the Arabian Peninsula in the Middle East; it ...

. Different authors have habits which can be reflected in their use of letters.

Hemingway Ernest Miller Hemingway (July 21, 1899 – July 2, 1961) was an American novelist, short-story writer, and journalist. His economical and understated style—which he termed the iceberg theory—had a strong influence on 20th-century fi ...

's writing style, for example, is visibly different from

Faulkner William Cuthbert Faulkner (; September 25, 1897 – July 6, 1962) was an American writer known for his novels and short stories set in the fictional Yoknapatawpha County, based on Lafayette County, Mississippi, where Faulkner spent most of ...

's. Letter,

bigram A bigram or digram is a sequence of two adjacent elements from a string of tokens, which are typically letters, syllables, or words. A bigram is an ''n''-gram for ''n''=2. The frequency distribution of every bigram in a string is commonly used f ...

trigram Trigrams are a special case of the ''n''-gram, where ''n'' is 3. They are often used in natural language processing for performing statistical analysis of texts and in cryptography for control and use of ciphers and codes. Frequency Contex ...

, word frequencies, word length, and sentence length can be calculated for specific authors, and used to prove or disprove authorship of texts, even for authors whose styles are not so divergent. Accurate average letter frequencies can only be gleaned by analyzing a large amount of representative text. With the availability of modern computing and collections of large

text corpora In linguistics, a corpus (plural ''corpora'') or text corpus is a language resource consisting of a large and structured set of texts (nowadays usually electronically stored and processed). In corpus linguistics, they are used to do statistical ...

, such calculations are easily made. Examples can be drawn from a variety of sources (press reporting, religious texts, scientific texts and general fiction) and there are differences especially for general fiction with the position of and , with becoming more common. Also, to note that different dialects of a language will also affect a letter's frequency. For example, an author in the United States would produce something in which is more common than an author in the United Kingdom writing on the same topic: words like "analyze", "apologize", and "recognize" contain the letter in American English, whereas the same words are spelled "analyse", "apologise", and "recognise" in British English. This would highly affect the frequency of the letter as it is a rarely used letter by British speakers in the English language. The "top twelve" letters constitute about 80% of the total usage. The "top eight" letters constitute about 65% of the total usage. Letter frequency as a function of rank can be fitted well by several rank functions, with the two-parameter Cocho/Beta rank function being the best. Another rank function with no adjustable free parameter also fits the letter frequency distribution reasonably well (the same function has been used to fit the amino acid frequency in protein sequences.) A spy using the

VIC cipher Vic (; es, Vic or Pancracio Celdrán (2004). Diccionario de topónimos españoles y sus gentilicios (5ª edición). Madrid: Espasa Calpe. p. 843. ISBN 978-84-670-3054-9. «Vic o Vich (viquense, vigitano, vigatán, ausense, ausetano, ausonense): ...

or some other cipher based on a straddling checkerboard typically uses a mnemonic such as "a sin to err" (dropping the second "r") or "at one sir" to remember the top eight characters.

Relative frequencies of letters in the English language

There are three ways to count letter frequency that result in very different charts for common letters. The first method, used in the chart below, is to count letter frequency in root words of a dictionary. The second is to include all word variants when counting, such as "abstracts", "abstracted" and "abstracting" and not just the root word of "abstract". This system results in letters like appearing much more frequently, such as when counting letters from lists of the most used English words on the Internet. A final variant is to count letters based on their frequency of use in actual texts, resulting in certain letter combinations like becoming more common due to the frequent use of common words like "the", "then", "both", "this", etc. Absolute usage frequency measures like this are used when creating keyboard layouts or letter frequencies in old fashioned printing presses. An analysis of entries in the Concise Oxford dictionary, ignoring frequency of word use, gives an order of "EARIOTNSLCUDPMHGBFYWKVXZJQ". The letter-frequency table below is taken from Pavel Mička's website, which cites Robert Lewand's ''Cryptological Mathematics''. According to Lewand, arranged from most to least common in appearance, the letters are: etaoinshrdlcumwfgypbvkjxqz. Lewand's ordering differs slightly from others, such as Cornell University Math Explorer's Project, which produced a table after measuring 40,000 words. In English, the space is slightly more frequent than the top letter () and the non-alphabetic characters (digits, punctuation, etc.) collectively occupy the fourth position (having already included the space) between and .

Relative frequencies of the first letters of a word in English language

The frequency of the first letters of words or names is helpful in pre-assigning space in physical files and indexes. Given 26

filing cabinet A filing cabinet (or sometimes file cabinet in American English) is a piece of office furniture for storing paper documents in file folders. In the most simple context, it is an enclosure for drawers in which items are stored. The two most comm ...

drawers, rather than a 1:1 assignment of one drawer to one letter of the alphabet, it is often useful to use a more equal-frequency-letter code by assigning several low-frequency letters to the same drawer (often one drawer is labeled VWXYZ), and to split up the most-frequent initial letters () into several drawers (often 6 drawers Aa-An, Ao-Az, Ca-Cj, Ck-Cz, Sa-Si, Sj-Sz). The same system is used in some multi-volume works such as some encyclopedias. Cutter numbers, another mapping of names to a more equal-frequency code, are used in some libraries. Both the overall letter distribution and the word-initial letter distribution approximately match the Zipf distribution and even more closely match the

Yule distribution Yule, actually Yuletide ("Yule time") is a festival observed by the historical Germanic peoples, later undergoing Christianised reformulation resulting in the now better-known Christmastide. The earliest references to Yule are by way of indig ...

. Often the frequency distribution of the first digit in each datum is significantly different from the overall frequency of all the digits in a set of numeric data, see

Benford's law Benford's law, also known as the Newcomb–Benford law, the law of anomalous numbers, or the first-digit law, is an observation that in many real-life sets of numerical data, the leading digit is likely to be small.Arno Berger and Theodore ...

for details. An analysis by

Peter Norvig Peter Norvig (born December 14, 1956) is an American computer scientist and Distinguished Education Fellow at the Stanford Institute for Human-Centered AI. He previously served as a director of research and search quality at Google. Norvig is t ...

on Google Books data determined, among other things, the frequency of first letters of English words. A June 2012 analysis using a text document containing all words in the English language exactly once, found to be the most common starting letter for words in the English language, followed by .

Relative frequencies of letters in other languages

*See İ and

dotless I I, or ı, called dotless I, is a letter used in the Latin-script alphabets of Azerbaijani, Crimean Tatar, Gagauz, Kazakh, Tatar, Kyrgyz, and Turkish. It commonly represents the close back unrounded vowel , except in Kazakh where it represen ...

. The figure below illustrates the frequency distributions of the 26 most common Latin letters across some languages. All of these languages use a similar 25+ character alphabet. Based on these tables, the ' etaoin shrdlu'-equivalent results for each language is as follows: *French: ''; (Indo-European: Italic; traditionally, 'esartinulop' is used, in part for its ease of pronunciationPerec, Georges; ''Alphabets''; Éditions Galilée, 1976) *Spanish: ''; (Indo-European: Italic) *Portuguese: '' (Indo-European: Italic) *Italian: ''; (Indo-European: Italic) *Esperanto: '' (artificial language – lexic influenced by Indo-European languages, Romance, Germanic mostly) *German: ''; (Indo-European: Germanic) *Swedish: ''; (Indo-European: Germanic) *Turkish: ''; (Turkic) *Dutch: ''; (Indo-European: Germanic) *Polish: ''; (Indo-European: Balto-Slavic) *Danish: ''; (Indo-European: Germanic) *Icelandic: ''; (Indo-European: Germanic) *Finnish: ''; (Uralic: Finnic) *Czech: ''; (Indo-European: Balto-Slavic)

Explanatory notes

References

External links

* * * *
Letter frequency
��simia.net

Useful tables

Useful tables for single letter, digram, trigram, tetragram, and pentagram frequencies based on 20,000 words that take into account word-length and letter-position combinations for words 3 to 7 letters in length: * * * * {{cite journal , last1=Mayzner , first1=M.S. , last2=Tresselt , first2=M.E. , last3=Wolin , first3=B.R. , title=Tables of pentagram frequency counts for various word-length and letter-position combinations , journal=Psychonomic Monograph Supplements , volume=1 , issue=5 , pages=144–190 , year=1965 Cryptography Frequency distribution Grammatology Phonology Quantitative linguistics